[v1] remove v0 code #344

yannicks1 · 2025-07-29T14:15:44Z

[v1] remove v0 code

Now as we have v1 support for embedding models (#277 ), we can finally delete the v0 code.
Note: for decoder models v0 support was depreciated some time ago.

Signed-off-by: Yannick Schnider <[email protected]>

github-actions · 2025-07-29T14:17:07Z

👋 Hi! Thank you for contributing to vLLM support on Spyre.
Just a reminder: Make sure that your code passes all the linting checks, otherwise your PR won't be able to be merged. To do so, first install the linting requirements, then run format.sh and commit the changes. This can be done with uv directly:

uv sync --frozen --group lint --active --inexact

Or this can be done with pip:

uv pip compile --group lint > requirements-lint.txt
pip install -r requirements-lint.txt
bash format.sh

Now you are good to go 🚀

joerunde · 2025-07-29T14:53:57Z

vllm_spyre/platform.py

-        if is_decoder and not envs.VLLM_USE_V1:
-            raise ValueError("Decoder models are only supported on v1")
+        if not envs.VLLM_USE_V1:
+            raise ValueError("vllm-spyre is only supported with vLLM v1")


Suggested change

raise ValueError("vllm-spyre is only supported with vLLM v1")

raise ValueError("vllm-spyre is only supported with vLLM v1. Please set VLLM_USE_V1=1")

joerunde · 2025-07-29T14:56:21Z

tests/e2e/test_spyre_embeddings.py


    monkeypatch.setenv("VLLM_SPYRE_DYNAMO_BACKEND", backend)
-    monkeypatch.setenv("VLLM_USE_V1", "1" if vllm_version == "V1" else "0")
+    monkeypatch.setenv("VLLM_USE_V1", "1")


Maybe we can replace all of these with one os.environ["VLLM_USE_V1"] = 1 in conftest.py?

But this env var is not needed anymore right. Should we not add an assert in the llm engine to verify that it is an instance of the the V1 Engine class?

we already have this check in the platform.py. Don't you think that is enough and we can safely remove all monkeypatch.setenv("VLLM_USE_V1", "1")?

I think at some point envs.VLLM_USE_V1 will be removed. Also, this flag doesn't make sure that the current vLLM instance is not running as V0 as vLLM currently can fallback to V0 if VLLM_USE_V1 is unset.

Can we check this flag with os.getenv instead? In this way the plugin won't crash when envs.VLLM_USE_V1 is removed upstream

We have VLLM_USE_V1 in other places in the code. These will have to be removed anyway when the envs.VLLM_USE_V1 is removed upstream...

But aren't they removed on this PR?

I saw that the only remaining occurrences were for a hack testing both v0 and v1 engines. As we don't use that anymore, I removed it and incorporated you check via os.getenv. see this commit.

maxdebayser · 2025-07-29T18:04:10Z

Nice!

Signed-off-by: Yannick Schnider <[email protected]>

yannicks1 · 2025-07-29T20:11:36Z

I did a very radical change and removed all lines where we set VLLM_USE_V1=1. We don't support any v0 code anymore in vllm-spyre. vllm enables v1 by default here, we do a check in platform.py here and tell the user to set it if it is unset for any reason. IMO this is safe enough. What do you think @joerunde @maxdebayser ?

joerunde · 2025-07-29T20:48:33Z

Yeah I think this is good enough then, this prevents anybody from running with VLLM_USE_V1=0

joerunde · 2025-07-29T20:49:24Z

@maxdebayser do we need to do a performance comparison of embeddings on v0 vs v1 first before deleting? Or are we good to go?

Signed-off-by: Yannick Schnider <[email protected]>

maxdebayser · 2025-07-30T13:21:42Z

@joerunde , I think we're good to go, I can run the V0 tests on a frozen version.

waleedqk · 2025-07-30T14:16:20Z

bot:test
MARKERS="spyre"

waleedqk · 2025-07-30T14:32:05Z

bot:test
MARKERS="spyre"

waleedqk · 2025-07-30T14:45:35Z

bot:test
MARKERS="spyre"

Signed-off-by: Yannick Schnider <[email protected]>

waleedqk · 2025-07-30T16:09:27Z

bot:test
MARKERS="spyre and not quantized and not multi and not cb"

waleedqk · 2025-07-30T17:36:21Z

bot:test
MARKERS="spyre and not quantized and not multi and not cb"

prashantgupta24 · 2025-07-30T17:57:44Z

docs/user_guide/supported_features.md

 | Speculative Decoding          |   🗓️   |      |
 | Guided Decoding               |   🗓️   |      |
-| Pooling                       |   ⚠️   | Works with V0. V1 still being developed in vLLM [vllm#18052](https://github.com/vllm-project/vllm/issues/18052) |
+| Pooling                       |   ✅   |      |


We already have Embedding models at the end of this table - is that still needed?

question for @maxdebayser

Since we don't support all pooling applications, I think it's better to remove this and leave just Embedding below.

Signed-off-by: Yannick Schnider <[email protected]>

joerunde

negative diff let's goooo

Signed-off-by: Yannick Schnider <[email protected]>

@maxdebayser

### [docs] remove pooling models from supported features Following up a (late) discussion in #344 about removing the pooling models from the list of supported models as not all all pooling applications are supported, and we already have embedding models in that list (see comment by @maxdebayser : [link](https://github.com/vllm-project/vllm-spyre/pull/344/files#r2247822334)) --------- Signed-off-by: Yannick Schnider <[email protected]>

prune v0 code

0fb0af8

Signed-off-by: Yannick Schnider <[email protected]>

joerunde reviewed Jul 29, 2025

View reviewed changes

Merge branch 'main' into ysc-prune-v0

5b7db09

rafvasq mentioned this pull request Jul 29, 2025

[Tools][CICD] Add/update linting scripts, actions foundation-model-stack/aiu-fms-testing-utils#94

Closed

yannicks1 added 2 commits July 29, 2025 19:53

update log

ca82ba9

Signed-off-by: Yannick Schnider <[email protected]>

don't set 'VLLM_USE_V1=1' all over the place.

d14abb9

Signed-off-by: Yannick Schnider <[email protected]>

yannicks1 marked this pull request as ready for review July 29, 2025 20:12

yannicks1 requested review from nikolaospapandreou, prashantgupta24, rafvasq, sducouedic and tdoublep as code owners July 29, 2025 20:12

yannicks1 self-assigned this Jul 29, 2025

yannicks1 added 2 commits July 29, 2025 21:05

update docs

2d8170a

Signed-off-by: Yannick Schnider <[email protected]>

Merge branch 'main' into ysc-prune-v0

ea64208

Signed-off-by: Yannick Schnider <[email protected]>

check via os.getenv() instead of envs

a286247

Signed-off-by: Yannick Schnider <[email protected]>

prashantgupta24 reviewed Jul 30, 2025

View reviewed changes

Merge branch 'main' into ysc-prune-v0

4327681

Signed-off-by: Yannick Schnider <[email protected]>

fix fmt

1229106

Signed-off-by: Yannick Schnider <[email protected]>

joerunde approved these changes Jul 31, 2025

View reviewed changes

remove v1 marker

f32878e

Signed-off-by: Yannick Schnider <[email protected]>

yannicks1 enabled auto-merge (squash) July 31, 2025 14:21

github-actions bot added the ready label Jul 31, 2025

yannicks1 merged commit 8e7d565 into main Jul 31, 2025
17 of 19 checks passed

yannicks1 deleted the ysc-prune-v0 branch July 31, 2025 15:28

yannicks1 mentioned this pull request Aug 4, 2025

[docs] remove pooling models from supported features #358

Merged

	raise ValueError("vllm-spyre is only supported with vLLM v1")
	raise ValueError("vllm-spyre is only supported with vLLM v1. Please set VLLM_USE_V1=1")

[v1] remove v0 code #344

[v1] remove v0 code #344

Uh oh!

Conversation

yannicks1 commented Jul 29, 2025

[v1] remove v0 code

Uh oh!

github-actions bot commented Jul 29, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

maxdebayser Jul 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

maxdebayser Jul 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

maxdebayser commented Jul 29, 2025

Uh oh!

yannicks1 commented Jul 29, 2025

Uh oh!

joerunde commented Jul 29, 2025

Uh oh!

joerunde commented Jul 29, 2025

Uh oh!

maxdebayser commented Jul 30, 2025

Uh oh!

waleedqk commented Jul 30, 2025

Uh oh!

waleedqk commented Jul 30, 2025

Uh oh!

waleedqk commented Jul 30, 2025

Uh oh!

waleedqk commented Jul 30, 2025

Uh oh!

waleedqk commented Jul 30, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joerunde left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

maxdebayser Jul 29, 2025 •

edited

Loading

maxdebayser Jul 29, 2025 •

edited

Loading